Repurposing existing animated content

ABSTRACT

Systems and methods for automatically animating a character based on an existing corpus of animation are described. The character may be from a previously produced feature animated film, and the data used for training may be the data used to animate the character in the film. A low-dimensional embedding for subsets of the existing animation corresponding to different semantic labels may be learned by mapping high-dimensional rig control parameters to a latent space. A particle model may be used to move within the latent space, thereby generating novel animations corresponding to the space&#39;s semantic label, such as a pose. Bridges may link a first pose of a first model within the latent space that is similar to a second pose of a second model of the space. Animations corresponding to transitions between semantic labels may be generated by creating animation paths that traverse a bridge from one model into another.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/409,479, entitled “REPURPOSING EXISTING ANIMATED CONTENT,” filed Jan.18, 2017, which in turn claims priority under 35 U.S.C. § 119(e) to U.S.Provisional Patent Application Ser. No. 62/280,654, entitled“REPURPOSING HAND ANIMATION FOR INTERACTIVE APPLICATIONS,” filed Jan.19, 2016. The contents of each of these applications are herebyincorporated by reference in their entireties.

BACKGROUND 1. Field

The present disclosure relates to computer-generated animation and, morespecifically, to repurposing existing animated content of a character tosynthesize new animated content of the character.

2. Related Art

A computer-generated feature animation is typically a labor- andtime-intensive process that results in the creation of characters withcompelling and unique personalities. However, because of the labor- andtime-intensive nature of computer-generated feature animation, takingone of these characters and synthesizing new content consisting ofcompelling and expressive animation of the character presents achallenge.

Statistical methods have been used to analyze and synthesize new motiondata (e.g., as described in Brand and Hertzmann 2000, Mukai and Kuriyama2005, and Lau et al. 2009). In particular, the Gaussian Process LatentVariable Model (GPLVM) (e.g., as described in Lawrence 2006) has beenused for a number of applications directed to animation, such assatisfying constraints, tracking human motion (e.g., as described inGrochow et al. 2004, Urtasun et al. 2005, and Wang et al. 2008), orproviding interactive control (e.g., as described in Ye and Liu 2010 andLevine et al. 2012). The GPLVM is used to reduce the dimension of themotion data and to create a statistical model of the animation.

However, while GPLVM tends to keep far data separated in a reduceddimensional space, it makes no effort to keep similar data points closetogether. Thus, to address this shortcoming, modifications to the GPLVMhave been proposed to make it better suited for modeling motion data byaddressing this limitation of the model. For example, back constraints(e.g., as described in Lawrence and Quiflonero Candela 2006) have beenapplied to the GPLVM to preserve local distances. For another example,dynamic models (e.g., as described in Wang et al. 2006 and Lawrence2007) have been introduced to model the time dependencies in animationdata. For another example, a connectivity prior (e.g., as described inLevine et al. 2012) has been proposed to ensure a high degree ofconnectivity among the animation data embedded in the low-dimensionallatent space.

Another shortcoming of the GPLVM is that the prior methods that modelanimation data using a GPLVM have only been used for full-body motioncapture data. Similar techniques have not been applied to manuallycreated animation for a film-quality character. A key difference betweenmotion capture data and manually created film-quality animation is thatthe manually created animation from a film-quality animation lies in asignificantly higher dimensional space than the motion capture data.

Furthermore, data-driven approaches to character control and animationsynthesis have focused only on full-body tasks, which are based onmotion graphs (e.g., as described in Kovar et al. 2002, Lee et al. 2002,Treuille et al. 2007, Lo and Zwicker 2008, and Lee et al. 2009). Thesemethods use a graph structure to describe how motion clips from alibrary can be connected and reordered to accomplish a task. However,while these approaches perform well with a large training set, smallerdata sets are not well-suited for motion graphs because of a lack ofvariety and transitions in the motions.

Similarly, other existing methods for character control includedata-driven and physics-based approaches (e.g., as described in Coros etal. 2009, Muico et al. 2009, Levine et al. 2012, and Tan et al. 2014)are applied to full-body human motion or hand motion (e.g., as describedin Andrews and Kry 2013). Thus, the tasks that the controllers aretrained for can be quantifiably measured, such as locomotion or reachingtasks. However, existing methods do not animate a non-human character'sface because tasks for facial animation are difficult to quantify.

Facial animation of non-human characters can be controlled byre-targeting recorded expressions. A commonly used method is blendshapemapping (e.g., as described in Buck et al. 2000, Chai et al. 2003, Seolet al. 2011, Bouaziz et al. 2013, and Cao et al. 2013), which mapsexpressions from an input model onto corresponding expressions from thetarget character. Then, motion is generated by blending between thedifferent facial shapes of the character. This approach uses an inputmodel, such as a video recording of a human, to drive the animation ofthe character. Blendshape mapping approaches, however, control facialanimation with recordings of a model. In addition, blendshape mappingapproaches require that the character's face be animated withblendshapes.

Lastly, as is well-known in the field, animated characters arecontrolled through an underlying rig, which deforms a surface mesh thatdefines a character. A variety of methods exist to map a character's rigcontrols to deformations of the surface mesh (e.g., as described in Barr1984, Sederberg and Parry 1986, Magnenat-Thalmann et al. 1988, Singh andFiume 1998, and Lewis et al. 2000). However, a technique that does notmake assumptions about rig controls, and thus does not depend on animplementation of a particular type of mapping method, is needed.

SUMMARY

In one exemplary embodiment, new animated content of a character isrendered based on existing animated content of the character. Inparticular, a first set of rig parameters corresponding to a firstplurality of frames of the existing animated content of the character isreceived. A second set of rig parameters corresponding to a secondplurality of frames of the existing animated content of the character isreceived. The first and second sets of rig parameters have a firstdimension. A first set of model parameters is generated based on thefirst set of rig parameters. A second set of model parameters isgenerated based on the second set of rig parameters. The first andsecond sets of model parameters have a second dimension smaller than thefirst dimension. A latent space is generated that comprises a firstsemantic model, a second semantic model, and a bridge. The latent spacehas the second dimension. The first semantic model is generated based onthe first set of model parameters. The second semantic model isgenerated based on the second set of model parameters. The bridge linksthe first semantic model and the second semantic model. The new animatedcontent of the character is generated based on a location within thelatent space.

BRIEF DESCRIPTION OF THE FIGURES

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawings will be provided by the Office upon request and paymentof the necessary fee.

The present application can be best understood by reference to thefollowing description taken in conjunction with the accompanying drawingfigures, in which like parts may be referred to by like numerals.

FIG. 1 illustrates exemplary existing animated content of a character.

FIG. 2 illustrates exemplary new animated content of the characterrendered based on the existing content.

FIG. 3 illustrates an exemplary process of rendering new animatedcontent from existing animated content.

FIG. 4 illustrates an exemplary three-dimensional latent space learnedfrom a training set of examples of a roar of the character.

FIG. 5 illustrates an exemplary cross-section plot of a pose matchingoptimization function.

FIG. 6 illustrates four examples of best-matching poses identifiedbetween two different models within the latent space.

FIG. 7 illustrates exemplary particle trajectories through the latentspace.

FIG. 8 illustrates an exemplary computing system.

DETAILED DESCRIPTION

The following description is presented to enable a person of ordinaryskill in the art to make and use the various embodiments. Descriptionsof specific devices, techniques, and applications are provided only asexamples. Various modifications to the examples described herein will bereadily apparent to those of ordinary skill in the art, and the generalprinciples defined herein may be applied to other examples andapplications without departing from the spirit and scope of the presenttechnology. Thus, the disclosed technology is not intended to be limitedto the examples described herein and shown, but is to be accorded thescope consistent with the claims.

For descriptive purposes, throughout this disclosure, software, softwaremodules, software objects, and the like may be described as performingvarious functions. One of ordinary skill in the art, however, willrecognize that software may not actively perform any function andinstead may include instructions that are executable on a computerprocessor. As such, although software may be described herein asperforming a function, it should be appreciated that a computerprocessor or other computing device may typically perform thosefunctions attributed herein to software modules or objects by executingcomputer instructions provided by the software modules or objects.

As noted above, creating characters with compelling and uniquepersonalities for a computer-generated feature animation can be laborand time intensive. Thus, as described below, existing animated contentof a character can be repurposed by mapping rig control parameters to alatent space in order to synthesize new animated content of thecharacter.

For example, FIG. 1 depicts existing animated content 102-108 ofToothless from the feature animated film How to Train Your Dragon 2.FIG. 2 depicts newly synthesized animation of Toothless generated usingexemplary process 300 described below and depicted in FIG. 3.

In FIG. 3, at block 302, a first set of rig parameters corresponding toa first plurality of frames of the existing animated content of thecharacter is received. At block 304, a second set of rig parameterscorresponding to a second plurality of frames of the existing animatedcontent of the character is received. The first and second sets of rigparameters have a first dimension.

In some embodiments, the first plurality of frames is associated with afirst pose of the character and the second plurality of frames isassociated with a second pose of the character. The second pose isdifferent than the first pose. The first pose can comprise a particularfacial expression of the character.

At block 306, a first set of model parameters is generated based on thefirst set of rig parameters. At block 308, a second set of modelparameters is generated based on the second set of rig parameters. Thefirst and second sets of model parameters have a second dimension thatis smaller than the first dimension.

In some embodiments, a first set of intermediate space parameters and asecond set of intermediate space parameters are generated from the firstset of rig parameters and the second set of rig parameters,respectively. Then, the first set of model parameters and the second setof model parameters are generated based on the first set of rigparameters and the second set of rig parameters, respectively.

In some embodiments, a first set of scaled parameters and a second setof scaled parameters are generated from the first set of rig parametersand the second set of rig parameters, respectively. Then, the first setof intermediate space parameters and the second set of intermediatespace parameters are generated based on the first set of scaledparameters and the second set of scaled parameters, respectively.

In some embodiments, generating a set of model parameters from a set ofintermediate space parameters comprises applying a non-lineardimensionality reduction algorithm to the set of intermediate spaceparameters. In some embodiments, this non-linear dimensionalityreduction algorithm is a Gaussian process latent variable model (GPLVM).In some embodiments, generating the set of intermediate space parametersfrom a set of scaled parameters comprises applying a lineardimensionality reduction algorithm to the set of scaled parameters. Insome embodiments, this linear dimensionality reduction algorithm isprincipal component analysis.

At block 310, a latent space that comprises a first semantic model, asecond semantic model, and a bridge is generated. The latent space has adimension equal to the second dimension. The first semantic model isgenerated based on the first set of model parameters. The secondsemantic model is generated based on the second set of model parameters.The bridge links the first semantic model and the second semantic model.

In some embodiments, creating the bridge comprises identifying a firsttransition point within the first semantic model, identifying a secondtransition point within the second semantic model, and linking the firsttransition point with the second transition point. In some examples, thefirst transition point and the second transition point are identifiedbased on similarities between a frame associated with the firsttransition point and a frame associated with the second transitionpoint. In some embodiments, the first transition point and the secondtransition point are identified using a pose-matching optimizationfunction.

At block 312, the new animated content of the character is renderedbased on a location within the latent space. In some embodiments,rendering the new animated content of the character based on thelocation within the latent space comprises determining a model parameterthat corresponds to the location within the latent space, determining arig parameter that corresponds to the model parameter, and thenrendering a frame of animation associated with the determined rigparameter. In some embodiments, rendering the new animated content ofthe character based on the location within the latent space comprisesdetermining a model parameter that corresponds to the location withinthe latent space, determining an intermediate space parameter thatcorresponds to the model parameter, determining a scaled parameter thatcorresponds to the intermediate space parameter, determining a rigparameter that corresponds to the scaled parameter, and then rendering aframe of animation associated with the determined rig parameter.

In some embodiments, a particle model is used to move a particle withinthe latent space, and the rendered new animated content of the charactercorresponds to the particle's location (i.e., position) within thelatent space as the particle is moved within the latent space. Theparticle is moved in accordance with an animation curve embedded withinthe latent space, and is moved between two or more different animationcurves via one or more bridges between the animation curves.

In some embodiments, it is determined whether the particle is not movedfor at least a predetermined amount of time. The character may be“idling” when the particle is not moved for at least the predeterminedamount of time. In some examples, if it is determined that the particleis not moved for at least the predetermined amount of time, the particleis moved around a hover point within the latent space, and the particleis not moved beyond a predetermined distance from the hover point. A“hovering” particle adds variety to the idling pose of the character.

Some of the particulars of process 300, including the mathematicsinvolved in process 300, are set forth in more detail below.

1.1 Scaling Rig Controls

In one exemplary embodiment, given a large set of training animation ofthe character, represented as a sequence of rig control parameters, amapping is generated between a low dimensional latent space and the rigparameters. The sequence of rig control parameters correspond to asequence of frames of the animated character. For example, FIG. 1illustrates four frames 102-108 of a synthesized roar animation of thecharacter Toothless the dragon 100 from the feature animated film How toTrain Your Dragon 2. The facial expression of Toothless 100 differs ineach frame, as each frame represents a specific stage of the roar. Thespecific pose (e.g., facial expression) of Toothless 100 shown in frames102-108 can be represented by a corresponding sequence of rig controlparameters.

The mapping between the low dimensional latent space and the rigparameters may be generated in three stages. First, each rig controlparameter in the training data is scaled to proportionally weight thecontrols to changes in the final mesh. Second, the training animationdata is reduced linearly. In some embodiments, the training animationdata is reduced linearly using Principal Component Analysis (PCA).Third, the training data is non-linearly reduced and mapped to a lowerdimensional latent space. In some embodiments, the Gaussian ProcessLatent Variable Model (GPLVM) is used to non-linearly reduce thetraining data to a three dimensional latent space. After an embedding ofthe training animation data in the latent space is found, any arbitrarypoint in the latent space may be mapped to values for the rig controls.This process of mapping from a set of rig control parameters to a lowdimensional latent space is described in greater detail below.

In some embodiments, it is assumed that the rig control parameters p,when evaluated, produce a surface mesh. The i^(th) vertex of this meshis given by the function e_(i)(p). In some embodiments, it is furtherassumed that the rig evaluation function e(p) is continuous, but noother assumptions are made. Thus, the evaluation function will be highlynonlinear.

Depending on how the evaluation function e(p) is defined, large changesin some rig control parameters can result in small changes in the outputsurface mesh while small changes in other parameters can result in largechanges in the mesh. Specifically, for some setting of the rig controlparameters p, the value

$\frac{\partial{e(p)}}{\partial p_{i}}$

may be large for the i^(th) rig control parameter, but the value

$\frac{\partial{e(p)}}{\partial p_{j}}$

may be small for some other rig control parameter. Thus, there existsthe possibility that some rig controls will have a very small effect onthe surface mesh but have a large variance across the training animationdata. If PCA is applied in this case, each component of the data will bescaled such that the principal axes of the transformation do not alignwith these controls with high variance but low influence on the mesh.

In some embodiments, to avoid the above-described situation, the rigcontrol parameters are scaled about the sample average to obtainz=W(p−p)+p, where W represents a diagonal matrix and w_(i) representsthe amount to scale the i^(th) rig control parameter by. W is chosensuch that a unit change in the scaled rig control parameter spaceapproximately corresponds with a unit change in the surface mesh.Specifically, for the i^(th) rig control parameter,

$\begin{matrix}{{{{\frac{\partial}{\partial z_{i}}{e\left( {{w^{- 1}\left( {z - \overset{\_}{p}} \right)} + \overset{\_}{p}} \right.}} = 1},}} & (1)\end{matrix}$

where z is any possible value of the scaled rig control parameters.Then, p=W⁻¹z and the chain rule may be used to find that

$\begin{matrix}{{{\frac{\partial{e(p)}}{\partial p_{i}}{\frac{\partial}{\partial z_{i}}\left\lbrack {{w_{i}^{- 1}\left( {z_{i} - {\overset{\_}{p}}_{i}} \right)} + {\overset{\_}{p}}_{i}} \right\rbrack}}} = 1.} & (2)\end{matrix}$

In some embodiments, Equation (2) is used to solve for the weights tofind that

$w_{i} = {{\frac{\partial{e(p)}}{\partial p_{i}}}.}$

Because e(p) is a generally nonlinear function, Equation (2) cannot besatisfied for all possible values of p for a fixed W. Instead, the normof the partial derivative can be approximated by evaluating the rig atthe sample mean p of the training data and at several points about themean. For rig control parameter i, a least squares error problem can beconstructed to approximate the norm of the partial derivative by

$\begin{matrix}{{\frac{\partial{e(p)}}{\partial p_{i}}} \approx {\underset{w}{\arg \; \min}{\sum\limits_{n = {- 2}}^{2}\left( {{{{e\left( \overset{\_}{p} \right)} - {e\left( {\overset{\_}{p} + {n\; \sigma_{i}}} \right)}}} - {w{{n\; \sigma_{i}}}}} \right)^{2}}}} & (3)\end{matrix}$

where σ_(i) is a vector with the sample standard deviation of the i^(th)rig control parameter in the i^(th) position and zeros elsewhere. Forexample, in Equation (3), the set of values n∈{−2, −1, 0, 1, 2} wasfound to produce good results. This least squares problem is solvedseparately for each w_(i).

1.2 Linear Dimensionality Reduction

Generally, a fully-rigged main character for a feature animation filmcan have on the order of thousands of rig controls. Depending on theamount and/or nature of the rig controls, some of these rig controls maynot be used in the training data. For example, some rig controls mayonly have a small, almost imperceptible effect on the animation and, assuch, may not be used in the training data. In some embodiments, toremove these minimally-impactful controls and simplify the data, thedimension of the training data is linearly reduced using PrincipalComponent Analysis (PCA). Consequently, small variations in the trainingdata are treated as noise and are removed. This initial linear reductionalso helps to improve the results of a subsequent non-lineardimensionality reduction, for example a Gaussian Process Latent VariableModel (GPLVM) based reduction, that may be applied to the training datalater on in the reduction process.

To demonstrate an exemplary use of PCA for linear dimensionalityreduction, let z represent the scaled rig control parameters of a singleframe of animation. There are D_(rig) control parameters and N totalnumber of frames of animation in the training set. The scaled animationdata may be represented as =[z₁, z₂, z₃ . . . , z^(N)], where z_(i)∈

^(D) ^(rig) is a column vector corresponding with the i^(th) frame ofanimation. The sample mean of the data Z is denoted as z. To compute theprincipal components of the training data, the mean is subtracted fromeach frame of animation. The singular value decomposition of thenormalized data can then be computed using

Z=UΣV ^(T),  (4)

where the matrix Z is the matrix Z with the sample mean subtracted fromeach column of the matrix.

The columns of the matrix U contain the principal components of thetraining data. The number of principal components d_(pca) to use can bedetermined by considering the explained variance of the model. Forexample, the explained variance can be given by

$\begin{matrix}{{{v(d)} = \frac{\sum_{i - 1}^{d}\sigma_{i}^{2}}{\sum_{i - 1}^{k}\sigma_{i}^{2}}},} & (5)\end{matrix}$

where σ_(i) ² is the i^(th) singular value of the normalized matrix Zand k is the rank of the matrix. In this example, the d_(pca) is chosensuch that ν(d_(pca))≈0.85. With the number of principal componentschosen, the transformation matrix T_(pca), which contains the firstd_(pca) columns of the matrix U, is defined. The training data may thenbe represented as a d_(pca)xn matrix Y given by

Y=T _(pca) ^(T) Z.  (6)

The difference between running PCA on the original and scaled rigcontrol parameters may be evaluated to determine the effect scaling theparameters have on the quality of the dimensionality reduction. Ifenough principal components are used to ensure that the explainedvariance is at or above 85%, there may be no discernible difference inquality of the animations between the scaled and original rig controlparameters, but the GPLVMs described in the following section tend toperform better with the scaled rig control parameters. The differencebetween the original rig control parameters and the compressed data,measured as ∥z−T_(pca)T_(pca) ^(T)z∥, may be much larger when using thescaled rig control parameters compared to the unsealed parameters. Whena small number of principal components are used, animations compressedwith the scaled rig control parameters are visually better than theanimations compressed with the unscaled data. Furthermore, the unscaledversion often contains objectively undesirable meshes, such as the jawof a character passing through the roof of its mouth. Therefore,quantitative comparisons in the rig control parameter space may not besufficient to evaluate the effectiveness of the disclosed method.

1.3 Nonlinear Dimensionality Reduction

In some embodiments, having generated the linearly reduced data in thematrix Y, a low-dimensional embedding through the use of a GaussianProcess Latent Variable Model (GPLVM) is computed. The GPLVM is agenerative, probabilistic model that is used to map nonlinearly the PCAtransformed data Y to a set of points X in a latent space of dimensiond_(gplvm) where d_(gplvm)<d_(pca). Dynamics in the latent space can bemodeled by placing a Gaussian process prior on the points X. Thisdynamics prior keeps temporally close data points close togetherspatially. Because the models are trained using multiple segments ofanimation, the GPLVM with a dynamics prior tends to keep separatesegments far apart in the latent space. This separation is caused by theGPLVM placing dissimilar frames of animation far apart without trying toplace similar frames near each other. Therefore, a connectivity priorcan be used to pull together similar frames of animation from separatesegments.

The GPLVM models the training data Y as the output of a Gaussian processfrom the low dimensional embedding of the points X. It is assumed thateach output of the Gaussian process is independent so that

log p(Y|X)=Σ_(i=1) ^(d) ^(pca) log N(y _(i),:|0,K _(x)),  (7)

which can be expressed as

$\begin{matrix}{{\log \; {p\left( Y \middle| X \right)}} = {{{- \frac{d_{pea}}{2}}{K_{x}}} - {\frac{1}{2}{{tr}\left( {K_{x}^{- 1}{YY}^{T}} \right)}} + {{const}.}}} & (8)\end{matrix}$

The i^(th) row of Y is denoted as y_(i,:). For the entries in the kernelmatrix K_(x), a radial basis function may be used. For example, theradial basis function is given by

$\begin{matrix}{{k_{X}\left( {x_{i},x_{j}} \right)} = {{\sigma_{rbf}^{2}{\exp\left( {{- \frac{1}{2l_{x}^{2}}}{{x_{i} - x_{j}}}^{2}} \right)}} + {\delta_{ij}{\sigma_{white}^{2}.}}}} & (9)\end{matrix}$

The kernel parameters σ_(rbf) ², σ_(white) ², and ι² are optimized whenthe GPLVM is trained.

In some embodiments, the input data is composed of multiple segments ofanimation, and the dynamics of each segment are modeled. For example, aGaussian process prior is placed on the latent points X. The input tothe Gaussian process is time t and an indicator s where s_(i)=jindicates that frame i is part of the j^(th) animation segment. Theindicator s is used in the kernel function to ensure that each segmentis independent of all the others. Further, the dynamics prior is givenby

ψ_(D)=(X,s,t)=Σ_(i=1) ^(d) ^(gplvm) log N(X _(i,:)|0,K _(st)),  (10)

which can be expressed as

ψ_(D)(X,s,t)=−½Σ_(i=1) ^(d) ^(gplvm) x _(i,:) ^(T) K _(st) ⁻¹ x_(i,:)+const.  (11)

As such, the entries in the kernel matrix K_(st) can be given by

$\begin{matrix}{{{k_{st}\left( {t_{i},t_{j},s_{i},s_{j}} \right)} = {{1\left\lbrack {s_{i} = s_{j}} \right\rbrack}\left( {{Ϛ_{rbf}^{2}{\exp\left( {- \frac{\left( {t_{i} - t_{j}} \right)^{2}}{2l_{t}^{2}}} \right)}} + {Ϛ_{white}^{2}\delta_{ij}}} \right)}},} & (12)\end{matrix}$

where 1[s_(i)=s_(j)] is the indicator function that returns 1 if s_(i)and s_(j) are equal and 0 otherwise.

In some embodiments, the connectivity prior allows for a modelling ofthe degree of connectivity amongst the latent points X by using graphdiffusion kernels. For example, if G is a fully connected graph with avertex for each training point in the latent space, then the edgeweights of the graph can be defined as

w(x _(i) ,x _(j))=∥x _(i) −x _(j)∥^(−p).  (13)

The weights represent the probability of a transition from x_(i) tox_(j). The value of p determines the preference for transitions betweencloser points. The connectivity prior defines a random walk process on Gand considers the latent points X to be well-connected if all points arelikely to be reached from any other point through the random walk.

In some embodiments, the connectivity of the latent points is modeled asa diffusion kernel using, for example, the negative normalized graphLaplacian H=T^(−1/2)T^(−1/2). The matrix T is diagonal with entriesT_(ii)=Σ_(j) w(x_(i), x_(j)), and the matrix L is defined as

$\begin{matrix}{L_{i}j\left\{ \begin{matrix}{\sum_{k}{w\left( {x_{i},x_{k}} \right)}} & {i = j} \\{- {w\left( {x_{i},x_{j}} \right)}} & {i \neq j}\end{matrix} \right.} & (14)\end{matrix}$

The graph diffusion kernel can be computed by applying the functionK^(d)=exp βH, where β is a user-defined diffusion rate. Having computedthe graph diffusion kernel, the connectivity prior can be calculatedusing the function:

ψ_(C)(x)=w _(c)Σ_(i)Σ_(j) K _(ij) ^(d),  (15)

where w_(c) is a user-defined parameter specifying the weight of theconnectivity prior.

Combining the dynamics and connectivity priors, the conditionalprobability of X may be expressed as

p(X|t,s)∝exp ψ_(D)(X,s,t)extψ _(C)(X)  (16)

The latent points X and the hyper-parameters σ_(rbf), σ_(white), andl_(x) can be estimated through maximum a posteriori (MAP) estimation. Assuch, the following can be maximized:

log p(X,σr _(bf),σ_(white) ,l _(x) |Y,S,t)=logp(Y|X)+ψ_(D)(X,s,t)+ψ_(C)(X).  (17)

To maximize Equation (17), a scaled conjugate gradient can be used. Theinitial guess for the latent points is the first d_(gplvm) rows of Y.The hyper-parameters for the dynamics prior are manually set and thesevalues are not optimized.

FIG. 4 illustrates a plot of nine animation curves embedded in anexemplary three-dimensional latent space 400. The exemplarythree-dimensional latent space 400 is learned for a training set of nineexamples of a roar of Toothless 100 with a total of 393 frames.Representative images of six of the nine different roars of Toothless100 that correspond to an animation curve embedded in the latent space400 are shown. Roar 402A depicted in image 402 corresponds to animationcurve 402B within the latent space 400. Roar 404A depicted in image 404corresponds to animation curve 404B within the latent space 400. Roar406A depicted in image 406 corresponds to animation curve 406B withinthe latent space 400. Roar 408A depicted in image 408 corresponds toanimation curve 408B within the latent space 400. Roar 410A depicted inimage 410 corresponds to animation curve 410B within the latent space400. Roar 412A depicted in image 412 corresponds to animation curve 412Bwithin the latent space 400.

1.4 Mapping to Rig Controls

Once a trained model is generated, rig control values can bereconstructed from a new point x′ in the latent space. In someembodiments, the most likely point in the d_(pca) dimensional spacegiven the new point and the GPLVM model is first identified. Next, thematrix of principal components is multiplied to the identified point inthe d_(pca) dimensional space to obtain the scaled rig controlparameters. Finally, the scaling factors are divided out and the mean isadded to each parameter.

The distribution of a new point y given the corresponding latent point xand the GPLVM model M is a Gaussian distribution, where

p(y|x,M)=N(y|YK _(x) ⁻¹ k _(x)(x),k _(x)(x,x)−k _(x)(x)^(T) K _(x) k_(x)(x)),  (18)

and where k_(x)(x) is a column vector whose i^(th) entry is given byk_(x)x_(i)=k_(x)(x_(i), x). Because the distribution is Gaussian, themost likely point in the d_(pca) dimensional space can be given by themean YK_(x) ⁻¹k_(x)(x). The product YK_(x) ⁻¹ is precomputed, thusallowing this pose reconstruction problem to run linearly to the size ofthe training data for the model.

2. Animation Synthesis in Latent Space

In some embodiments, new animations are synthesized by generating a newpath P=[x₁, x₂, . . . , x_(t)] through the latent space. The rig controlparameters for each point in the path is computed by mapping the pointfrom the latent space to the high dimensional rig control space. Becausethe latent space provides a continuous mapping, any smooth curve in thislow-dimensional space results in smooth animation curves for each rigcontrol parameter.

To synthesize a new path, a particle moving through the latent space canbe simulated, and its position over time can be tracked. In someexamples, the particle is controlled using a Lagrange multiplier methodto enforce constraints on the system. In some examples, if a path thatdoes not stray too far from a user-defined point is desired, aconstraint is defined to enforce this behavior. In some examples, to addvariations and noise to the path, a random force is applied. Theparticle simulation method works well for synthesizing facialanimations.

In order to achieve real-time performance, it can be helpful if thenumber of training points in the GPLVM is small. Thus, in someembodiments, in order to condense the overall number of training pointsin the GPLVM, the training data is divided into sufficiently smallsubsets of training data, where each subset of data corresponds with aspecific type of pose (e.g., a facial expression or action and/or abodily expression or action), such as a roar. As such, a separate GPLVMis trained on each subset of training data that corresponds with aspecific type of pose, with each subset of training data generating amodel within the latent space that corresponds with the specific type ofpose of its respective subset of training data. However, because theselatent spaces are separate, points from one model (e.g., correspondingto a first pose of the character) to another model (e.g., correspondingto the second pose of the character) have to be mapped. The mappingallows a particle to transition between models, which in turn allows forthe synthesis of poses across multiple subsets of the training datacorresponding to different types of poses of the character.

2.1 Particle Simulation

In some embodiments, curves in the latent space are synthesized bytracking the position of a particle in the latent space over time. Theinput to a simulation is a path p(t) that the particle follows throughtime. In some examples, two constraints are applied to the system and a“random” force to add noise to the path. The first constraint ensuresthat the particle does not move too far from the path. The secondconstraint ensures that the particle remains in areas of highprobability in the GPLVM. Because there may be instances where bothconstraints cannot be satisfied simultaneously, the path-followingconstraint is modeled as a hard constraint that must be satisfied, andthe other constraint is modeled as a soft constraint that may beviolated.

In some embodiments, given some path p(t) parametrized by time, it isrequired that the particle does not drift too far away from the curve.Thus, to enforce this requirement, the following inequality constraintis applied:

∥x−p(t)∥² −r ²≥0.  (19)

This inequality constraint ensures that the particle at location x stayswithin a distance r from the point p(t) at time t. Forward simulationwith this constraint can be computed using the Lagrange multipliermethod.

Further, a force F acting on the particle at time t is defined.Subsequently, the Lagrange multiplier method is applied to compute anadditional force F_(c) that is applied to the particle to ensure thatthe constraint is satisfied. For example, the constraint force is givenby F_(C)=λg, where g=x(t)−p(t), the multiplier λ for a particle of unitmass is given by

$\begin{matrix}{{\lambda \; \frac{{{- g^{T}}F} + G}{g^{T}g}},} & (20)\end{matrix}$

and the scalar G is given by

G=({dot over (x)}(t)−p{dot over (()}t))^(T)(x(t){dot over (−)}p{dot over(()}t))+2αg ^(T) x(t)−g ^(T)(p(t){dot over ())}∔½β²(g ^(T) g−r ²).  (21)

The parameters α and β are selected by the user to control how quickly asystem violating the constraints returns to a state satisfying them. Forexample, a β of β=α² can be set. In this case, the term F_(c) describedabove will apply a force to satisfy the equality constraint∥x(t)−p(t)∥2−r2=0. Further, to allow the particle to move freely withinthe radius around the target point, the term F_(c) can be constrained toonly point towards the target point p(t), which is accomplished bysetting λ=0 whenever λ>0.

The second constraint pushes the particle towards high probabilityregions in the latent space. The GPLVM provides a probabilitydistribution over the latent space p(x(t)|M), and this distribution isused to push the particle towards “probable” regions, which may providebetter reconstructed poses as compared to less probable regions of thelatent space. However, models trained with facial animations are capableof synthesizing reasonable poses from less likely regions of the latentspace. Generally, these lower probability poses do not contain visualdefects such as an overly stretched face or interpenetrating meshes.Therefore, keeping the particle in a high probability region is notcritical, and thus may be violated if necessary to satisfy the pathconstraint.

In some embodiments, the likelihood constraint is modeled as a forceapplied to the particle that points in the direction of the gradient ofthe PDF. The magnitude of the force can be determined by the value ofthe PDF evaluated at the particle's current location. For example, ifthe value is above some empirically chosen quantity v, the magnitude issmall, and if the value is below v, the magnitude is large. Theseproperties are modeled as a sigmoid function so that the force functionis continuous for numerical integration. As such, the magnitude can beexpressed as

$\begin{matrix}{{{S(t)} = {a\; \frac{1}{1 + {\exp\left( \frac{p\left( {x(t)} \middle| {M - v} \right.}{l} \right)}}}},} & (22)\end{matrix}$

and the constraint force can be expressed as

$\begin{matrix}{{F_{GPLVM}(t)} = {{S(t)}{\frac{\partial{p\left( {x(t)} \middle| M \right.}}{\partial x}/{{\frac{\partial{p\left( {x(t)} \middle| M \right.}}{\partial x}}.}}}} & (23)\end{matrix}$

The parameters a and l are defined by the user. The user can alsocontrol the magnitude of the force when the constraint is not satisfiedand how quickly the magnitude approaches a. Computing the partialderivatives of the Gaussian process takes time quadratic to the size ofthe training data. However, if the size of the training set is small,the computations can be done in real-time.

In some embodiments, in addition to these constraint forces, a randomforce F_(rand)(t) is applied to add variation to the particle's path.This force is modeled as a randomly drawn, zero-mean Gaussian process

F _(rand)(t)˜

(0,k(t,t′)).  (24)

Each component of F_(rand)(t) is independent of all others. Theco-variance function is given by

$\begin{matrix}{{{k\left( {t,t^{\prime}} \right)} = {\alpha \; {\exp\left( {{- \frac{1}{2\gamma}}\left( {t - t^{\prime}} \right)^{2}} \right)}}},} & (25)\end{matrix}$

where α and γ are user-defined parameters that control the magnitude andsmoothness of the random force. This random force adds noise andvariations to the particle's movement through the latent space. As such,there can be at least small variations in each repetition that aparticle follows a particular path. This allows for the synthesis ofunique animations with small but noticeable differences even along thesame path.

Further, the particle can be simulated forward in time using, forexample, a fourth-order Runge-Kutta integration method. A piecewiselinear function is used for the path p(t), which is defined by a set ofpoints [p₁, p₂, . . . p_(n)] such that p(t_(i))=p_(i) and t_(i) is thetime of the i^(th) frame of animation. Integration across multipleframes of animation is avoided to prevent integration overdiscontinuities in the piecewise path function p(t). Section 4.3, below,describes an exemplary process for defining p(t).

2.2 Mapping Between Models

In general, a large set of heterogeneous motions cannot be accuratelyembedded in a low dimensional (e.g., d≤5) latent space. Therefore, insome embodiments, the training data is divided into small subsets ofsimilar poses (e.g., similar facial expressions), and the embedding inthe latent space is computed for each subset separately. However, adrawback of training separate models is that synthesizing animationstransitioning amongst multiple models cannot be synthesized using aparticle simulation method without a continuous path between the models.That is, an animation that smoothly transitions from one pose to anotherpose cannot be synthesized under the particle simulation method withouta continuous path for the particle to follow between the differentmodels.

As such, in some embodiments, a path is created between two models M₁and M₂ by first computing a set S of corresponding points in both modelswithin the latent space. For example, a pair of matching points (x₁,x₂), where x₁∈ M₁ and x₂∈M₂, is included in S if ∥g(x₁; M₁)−g(x₂;M₂)∥²<ϵ, where g(x; M) is the function that maps x to the rig controlparameter space. This enables the identification of points within themodels M₁ and M₂ whose reconstructed poses are similar. The set ofmatching points identifies points in the two models, which are connectedto form bridges between the two models. For example, to create a curvethat moves between model M₁ to model M₂, a path in M₁ that ends at amatching point in S and a path that starts at the matching point in M₂is created to form a bridge.

In some embodiments, to identify a pair of matching points for models M₁and M₂, a point x₁∈M₁ is used as a basis for computing the reconstructedrig control parameters z₁=g(x₁; M₁). Then, the point z₁ is transformedby the linear dimensionality reduction specified by model M₂

ŷ ₁ =T ₂ ^(T)[W ₂(z ₁ −m ₂)],  (26)

where T₂ represents the first d principal components of the PCAtransformation given in model M₂, W₂ represents the diagonal matrix ofscale values for each component, and m₂ represents the mean of trainingdata used in model M₂.

Next, the point x₂ in the latent space of model M₂ is found, where

$\begin{matrix}{x_{2} = {\underset{x}{\arg \; \min}{{{\hat{y}}_{1} - {\underset{y}{\arg \; \max}\log \; {p\left( {\left. y \middle| x \right.,M_{2}} \right)}}}}^{2}}} & (27)\end{matrix}$

Because y_(i)=f(x)+ϵ, where ϵ is additive Gaussian white noise, themaximum of p(y|x, M₂) occurs when y=f* where f*=K*[K_(x)]⁻¹Y₂ is thenoise-free output for the test point x. Therefore, Equation (27) can bere-written as

$\begin{matrix}{x_{2} = {\underset{X}{\arg \; \min}{{{{\hat{y}}_{1} - {{K_{*}\left\lbrack K_{x} \right\rbrack}^{- 1}Y_{2}}}}^{2}.}}} & (28)\end{matrix}$

As such, the problem of finding the best matching x₂∈M₂ given the pointx₁∈M₁ can now be formulated as a nonlinear optimization problem, whichcan be solved using the scaled conjugate gradient algorithm. However,because the function is multi-modal, the optimization algorithm isevaluated multiple times with randomly selected initial values toattempt to find the global minimizer. Further, large steps are not takenduring the optimization routine because the gradient of the objectivefunction quickly goes to zero as x₂ moves away from the training pointsin the model as illustrated, for example, in the cross section plot 500of the objective function 502 depicted in FIG. 5. The pose matchingoptimization function 502 shows the nonlinear behavior of the functionand the flat regions as the function extends to the left and the right.

In some embodiments, in order to create an animation that transitionsbetween two or more models, a first curve can be generated in the firstmodel that ends at one of the precomputed transition points, and asecond curve can be generated in the second model that starts at thecorresponding transition point from the first model. Then, the animationmay be synthesized by reconstructing the poses along the two curves andrendering the animation from the second model right after the animationfrom the first model.

FIG. 6 illustrates four sample pairs 602, 604, 606, and 608 of thebest-matching poses of Toothless 100 found between two different models.In each pair, the pose on the left is generated from a model trained onanimations with grumpy-looking animations, and the pose on the right isgenerated from happy-looking animations. That is, in pair 602, pose 602Ais a grumpy-looking Toothless 100, and pose 602B is a happy-lookingToothless 100 that is a best-matching happy pose to grumpy pose 602A. Inpair 604, pose 604A is a grumpy-looking Toothless 100, and pose 604B isa happy-looking Toothless 100 that is a best-matching happy pose togrumpy pose 604A. In pair 606, pose 606A is a grumpy-looking Toothless100, and pose 606B is a happy-looking Toothless 100 that is abest-matching happy pose to grumpy pose 606A. In pair 608, pose 608A isa grumpy-looking Toothless 100, and pose 608B is a happy-lookingToothless 100 that is a best-matching happy pose to grumpy pose 608A.

However, as shown in FIG. 6, the poses reconstructed from matchinglatent points in two different models may not necessarily be identical.As a result, there can be a discontinuity in the animation at thetransition between the two models. In some embodiments, to overcome thisproblem, a short blend is performed between the two poses in the rigcontrol parameter space at the transition point.

2.3 Synthesis Control

In some embodiments, an interactive application with a set of commandsprovides intuitive control of the character's poses (e.g., facialexpressions). These commands provide control over the particular posethe character is expressing at a specific time during the synthesizedanimation. With these poses, the synthesis algorithm can generatetransitions between the poses and models specified in the commands. Fourexample commands, a MOVE command, an IDLE command, a TRANSITION command,and a PLAY SEGMENT command, are described in greater detail below.

First, an exemplary MOVE command is described. The MOVE command takes atarget point t in the latent space as input. The synthesized animationis controlled by moving the particle from its current position in thelatent space to the target point. This is accomplished by setting theparticle's path function p(t). Different processes may be used togenerate the path. In some examples, a straight line is created from thecurrent point to the target. In other examples, the shortest path isused in a weighted graph G of the training data, where the weights arecomputed by Equation (13) in the same way that they are used for theconnectivity prior. FIG. 7 illustrates a side-by-side comparison of theparticle trajectories through the latent space these two exampleprocesses. Plot 700 shows function 702 that is defined as a series ofstraight lines between points 704A-704E. By contrast, plot 710 showscorresponding points 714A-714E, but function 712 is instead defined asthe shortest path through the weighted graph. In some examples, thestart and end points may also be added as vertices in the graph.Further, in some examples, resulting path may be re-sampled so that

$\frac{\partial{p(t)}}{\partial t}$

is constant for all t. This ensures that the particle follows the pathat a consistent speed. Both of the described path-generating processescreate compelling animation. A difference between the two processes isthat the straight line path is shorter, and thus a particle followingthis path will reach the target in less time.

Second, an exemplary IDLE command is described. A character is said tobe “idling” if the character is not performing a particular action andis instead in a non-active (e.g., “neutral”) state. In such situations,it may be beneficial for the character to demonstrate an “idling”animation, where the pose (e.g., expression) of the character iscontrolled as it idles. As discussed, each set of training data consistsof animations having similar poses (e.g., expressions). For example, oneset may include only animation poses corresponding to a happy facialexpression. Thus, each model corresponds with a single expression, andthe type of expression of the character's face can be controlled byusing the appropriate model. As such, idling animations can besynthesized by selecting a point p in the latent space and letting theparticle “hover” around that point. Further, a radius r may be definedthat can be adjusted to control how far the particle is allowed to strayaway from point p. In some examples, this is accomplished by setting thepath function that the particle follows to p(t)=p for a range of timet∈T in which the particle is to hover around the point. In someexamples, to add variety to the animation, multiple latent points may beused. With multiple points, the synthesis is controlled by first pickinga point from the set to move to. Next, the particle hovers about thatpoint for a fixed or predetermined amount of time. When a new point isselected, the simulation repeats by moving to the new point and hoveringaround the new point.

Third, an exemplary TRANSITION command is described. The TRANSITIONcommand is used to generate a continuous animation when moving betweentwo models. In some embodiments, the TRANSITION command uses, incombination, the previously described MOVE and IDLE commands. Thus, totransition from model M₁ to model M₂, the particle is moved from itscurrent position in model M₁ to the nearest precomputed matching pointin the latent space. When the particle is close to the matching point,it “idles” about that point and the particle in M₂ also begins to “idle”about the corresponding matching point. The transition is completed byperforming a blend between the high-dimensional rig control parametersfrom the two models while the particles are idling.

Fourth, an exemplary PLAY SEGMENT command is described. In somesituations, playing a portion of (e.g., a segment of) an unmodifiedanimation directly from the training data may be desirable. In someembodiments, the unmodified animation is played by using the embeddingof the sequence in the latent space. The MOVE command can then be usedto position the particle near the starting pose of the animation. Whenthe particle is close enough to the starting pose of the animation, thesimulation is stopped, and the particle is moved along the path of theembedded animation. When moving the particle to the start, the radius ris adjusted to ensure that it has moved close to the start to avoiddiscontinuities when the animation segment is played.

4. Animation System

FIG. 8 illustrates an exemplary animation system 800 that can be used toimplement the animation synthesis process discussed above. The animationsynthesis process can be implemented, for example, in either hardware orin software stored on a non-transitory computer-readable storage medium.The system can be configured to generate, modify, and evaluate theanimation synthesis process for repurposing existing animated content ofa character, as well as external processes used to render acomputer-generated image of the character. The system can be furtherconfigured to receive input from a user and to display graphics, animage, or scene of an animation based on the animation synthesisprocess.

The animation system 800 can be configured to receive user input from aninput device 820. The input device 820 can be any device that receivesinput from the user and transmits it to the animation system 800. Forexample, the input device 820 can be a keyboard, a mouse, a tablet, astylus, or the like. Those skilled in the art will recognize that othertypes of input devices can also be used.

The animation system 800 can be configured to output graphics, images,or animation to an output device 830. The output device 830 can includeany device that receives data from the animation system and presents thedata to the user. For example, the output device 830 may include aliquid crystal display, a set of light-emitting diodes, a projector, orthe like. Those skilled in the art will recognize that other types ofoutput devices can also be used.

The animation system 800 may further include a central processing unit802. The central processing unit may include one or more processingcores. The central processing unit 802 may be coupled to and able tocommunicate with the input device 820. Although the animation system 800is illustrated with one central processing unit 802, the animationsystem 800 may include multiple processing units. The animation system800 may also include a graphics processing unit 804. The graphicsprocessing unit 804 may be dedicated to processing graphics-relateddata. The graphics processing unit 804 may include a single processingcore or multiple processing cores. Although the animation system 800 isillustrated with one graphics processing unit 804, the animation system800 may include a plurality of graphics processing units. The centralprocessing unit 802 and/or the graphics processing unit 804 may becoupled to and be able to communicate data to the output device 830.

In one example, the animation system 800 may include one or moreprocessors and instructions stored in a non-transitory computer-readablestorage medium, such as a memory or storage device, that, when executedby the one or more processors, perform animation rendering using ananimation synthesis process, as described above. In the context of theembodiments described herein, a “non-transitory computer-readablestorage medium” can be any medium that can contain or store the programfor use by or in connection with the instruction execution system,apparatus, or device. The non-transitory computer-readable storagemedium can include, but is not limited to, an electronic, magnetic,optical, electromagnetic, infrared, or semiconductor system, apparatusor device, a portable computer diskette (magnetic), a random accessmemory (RAM) (magnetic), a read-only memory (ROM) (magnetic), anerasable programmable read-only memory (EPROM) (magnetic), a portableoptical disc such a CD, CD-R, CD-RW, DVD, DVD-R, or DVD-RW, or flashmemory such as compact flash cards, secured digital cards, USB memorydevices, memory sticks, and the like.

The animation system 800 may include volatile memory 806, which is anon-transitory computer-readable storage medium, in communication withthe central processing unit 802. The volatile memory 806 may include,for example, random access memory, such as dynamic random access memoryor static random access memory, or any other type of volatile memory.The volatile memory 806 may be used to store data or instructions duringthe operation of the animation system 800. Those skilled in the art willrecognize that other types of volatile memory can also be used.

The animation system 800 may also include non-volatile memory 808, whichis a non-transitory computer-readable storage medium, in communicationwith the central processing unit 802. The non-volatile memory 808 mayinclude flash memory, hard disks, magnetic storage devices, read-onlymemory, or the like. The non-volatile memory 808 may be used to storeanimation data, synthesis algorithm data, computer instructions, or anyother information. Those skilled in the art will recognize that othertypes of non-volatile memory can also be used.

The animation system 800 is not limited to the devices, configurations,and functionalities described above. For example, although a singlevolatile memory 806, non-volatile memory 808, central processing unit802, graphics processing unit 804, input device 820, and output device830 are illustrated, a plurality of any of these devices can beimplemented internal or external to the animation system 800. Inaddition, the animation system 800 may include a network access devicefor accessing information on a network, such as an internal network orthe Internet. Those skilled in the art will recognize that otherconfigurations of the animation system 800 can be used.

Various exemplary embodiments are described herein. Reference is made tothese examples in a non-limiting sense. They are provided to morebroadly illustrate applicable aspects of the disclosed technology.Various changes may be made and equivalents may be substituted withoutdeparting from the true spirit and scope of the various embodiments. Inaddition, many modifications may be made to adapt a particularsituation, material, composition of matter, process, process act(s), orstep(s) to the objective(s), spirit, or scope of the variousembodiments. Further, as will be appreciated by those with skill in theart, each of the individual variations described and illustrated hereinhas discrete components and features that may be readily separated fromor combined with the features of any of the other several embodimentswithout departing from the scope or spirit of the various embodiments.

1. A computer-implemented method for rendering new animated content of a character based on existing animated content of the character, the method comprising: accessing a latent space that comprises a first semantic model, a second semantic model, and a bridge, wherein: the first semantic model corresponds to a first plurality of frames of the existing animated content of the character and is generated based on a first set of model parameters, the second semantic model corresponds to a second plurality of frames of the existing animated content of the character and is generated based on a second set of model parameters, and the bridge links the first semantic model and the second semantic model; and rendering the character with a pose that is based on a location of a particle within the latent space, wherein the location of the particle within the latent space is controlled via user input, and wherein rendering the character with the pose that is based on the location of the particle within the latent space comprises: determining a new model parameter that corresponds to the location of the particle within the latent space; determining a new rig parameter that corresponds to the new model parameter; and rendering a new frame of animation associated with the determined new rig parameter.
 2. The computer-implemented method of claim 1, further comprising: applying a constraint to the particle, wherein the constraint constrains the particle to be within a predetermined distance from a path within the latent space.
 3. The computer-implemented method of claim 1, wherein the first plurality of frames of the existing animated content of the character corresponds to a first pose of the character, and wherein the second plurality of frames of the existing animated content of the character corresponds to a second pose of the character that is different from the first pose.
 4. The computer-implemented method of claim 3, wherein the first pose comprises a facial expression of the character.
 5. The computer-implemented method of claim 1, wherein rendering the character with the pose that is based on a location of the particle within the latent space further comprises: identifying a first transition point within the first semantic model; and identifying a second transition point within the second semantic model, wherein the bridge links the first transition point with the second transition point, and wherein the first transition point and the second transition point are identified using a pose-matching optimization function.
 6. The computer-implemented method of claim 5, wherein the first transition point and the second transition point are identified based on similarities between a first frame associated with the first transition point and a second frame associated with the second transition point.
 7. The computer-implemented method of claim 5, further comprising: receiving the user input; and in response to a determination that the user input is of a first type, moving the particle from the first transition point to the second transition point via the bridge.
 8. The computer-implemented method of claim 5, further comprising: receiving the user input; and in response to a determination that the user input is of a second type, moving the particle within the first semantic model without moving the particle from the first transition point to the second transition point via the bridge.
 9. The computer-implemented method of claim 1, wherein determining the new rig parameter that corresponds to the new model parameter further comprises: determining a new intermediate space parameter that corresponds to the new model parameter; determining a new scaled parameter that corresponds to the new intermediate space parameter; and determining the new rig parameter that corresponds to the new scaled parameter.
 10. The computer-implemented method of claim 9, wherein the new scaled parameter corresponding to the intermediate space parameter is based on a linear dimensionality reduction algorithm, and wherein the new intermediate space parameter corresponding to the new model parameter is based on a non-linear dimensionality reduction algorithm.
 11. The computer-implemented method of claim 10, wherein the linear dimensionality reduction algorithm is principal component analysis.
 12. The computer-implemented method of claim 10, wherein the non-linear dimensionality reduction algorithm is a Gaussian process latent variable model.
 13. The computer-implemented method of claim 1, further comprising: determining that the user input has not been detected for a predetermined amount of time; and in response to the determination that the user input has not been detected for a predetermined amount of time, rendering the character with an idling pose.
 14. The computer-implemented method of claim 1, further comprising: prior to accessing the latent space, generating the latent space, wherein generating the latent space comprises: receiving a first set of rig parameters corresponding to a first plurality of frames of the existing animated content of the character; receiving a second set of rig parameters corresponding to a second plurality of frames of the existing animated content of the character; generating a first set of model parameters based on the first set of rig parameters; generating a second set of model parameters based on the second set of rig parameters; and generating the latent space, wherein the first semantic model is generated based on the first set of model parameters and the second semantic model is generated based on the second set of model parameters.
 15. A non-transitory computer-readable storage medium storing one or more programs for rendering new animated content of a character based on existing animated content of the character, the one or more programs configured to be executed by one or more processors and including instructions for: accessing a latent space that comprises a first semantic model, a second semantic model, and a bridge, wherein: the first semantic model corresponds to a first plurality of frames of the existing animated content of the character and is generated based on a first set of model parameters, the second semantic model corresponds to a second plurality of frames of the existing animated content of the character and is generated based on a second set of model parameters, and the bridge links the first semantic model and the second semantic model; and rendering the character with a pose that is based on a location of a particle within the latent space, wherein the location of the particle within the latent space is controlled via user input, and wherein rendering the character with the pose that is based on the location of the particle within the latent space comprises: determining a new model parameter that corresponds to the location of the particle within the latent space; determining a new rig parameter that corresponds to the new model parameter; and rendering a new frame of animation associated with the determined new rig parameter.
 16. The non-transitory computer-readable storage medium of claim 15, wherein rendering the character with the pose that is based on a location of the particle within the latent space further comprises: identifying a first transition point within the first semantic model; and identifying a second transition point within the second semantic model, wherein the bridge links the first transition point with the second transition point, and wherein the first transition point and the second transition point are identified using a pose-matching optimization function.
 17. The non-transitory computer-readable storage medium of claim 15, wherein the one or more programs further include instructions for: prior to accessing the latent space, generating the latent space, wherein generating the latent space comprises: receiving a first set of rig parameters corresponding to a first plurality of frames of the existing animated content of the character; receiving a second set of rig parameters corresponding to a second plurality of frames of the existing animated content of the character; generating a first set of model parameters based on the first set of rig parameters; generating a second set of model parameters based on the second set of rig parameters; and generating the latent space, wherein the first semantic model is generated based on the first set of model parameters and the second semantic model is generated based on the second set of model parameters.
 18. An apparatus for rendering new animated content of a character based on existing animated content of the character, the apparatus comprising: one or more processors; and memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: accessing a latent space that comprises a first semantic model, a second semantic model, and a bridge, wherein: the first semantic model corresponds to a first plurality of frames of the existing animated content of the character and is generated based on a first set of model parameters, the second semantic model corresponds to a second plurality of frames of the existing animated content of the character and is generated based on a second set of model parameters, and the bridge links the first semantic model and the second semantic model; and rendering the character with a pose that is based on a location of a particle within the latent space, wherein the location of the particle within the latent space is controlled via user input, and wherein rendering the character with the pose that is based on the location of the particle within the latent space comprises: determining a new model parameter that corresponds to the location of the particle within the latent space; determining a new rig parameter that corresponds to the new model parameter; and rendering a new frame of animation associated with the determined new rig parameter.
 19. The apparatus of claim 18, wherein rendering the character with the pose that is based on a location of the particle within the latent space further comprises: identifying a first transition point within the first semantic model; and identifying a second transition point within the second semantic model, wherein the bridge links the first transition point with the second transition point, and wherein the first transition point and the second transition point are identified using a pose-matching optimization function.
 20. The apparatus of claim 18, wherein the one or more programs further include instructions for: prior to accessing the latent space, generating the latent space, wherein generating the latent space comprises: receiving a first set of rig parameters corresponding to a first plurality of frames of the existing animated content of the character; receiving a second set of rig parameters corresponding to a second plurality of frames of the existing animated content of the character; generating a first set of model parameters based on the first set of rig parameters; generating a second set of model parameters based on the second set of rig parameters; and generating the latent space, wherein the first semantic model is generated based on the first set of model parameters and the second semantic model is generated based on the second set of model parameters. 